The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model. This inspires us to design a powerful and efficient proxy task based on the Augmentation-Wise Weight Sharing (AWS) to form a fast yet accurate evaluation process in an elegant way.

Summary and Contributions: POST REBUTTAL: After reading the authors' response and discussing with other reviewers, I decided to keep my score. In this paper, the authors propose Augmentation-Wise Weight Sharing (AWS), a weight sharing strategy to efficiently search for data augmentation operations in image classification. AWS is based on a simple observation: data augmentation is more effective later in the training process, rather than earlier. Thus, AWS trains a single model for a while, and then only performs data augmentation search for a few last epochs, all starting from the trained shared weights. I think this is such a simple and elegant observation.

The recent progress on automatically searching augmentation policies has boosted the performance substantially for various tasks. A key component of automatic augmentation search is the evaluation process for a particular augmentation policy, which is utilized to return reward and usually runs thousands of times. A plain evaluation process, which includes full model training and validation, would be time-consuming. To achieve efficiency, many choose to sacrifice evaluation reliability for speed. In this paper, we dive into the dynamics of augmented training of the model.